npj Imaging

Magnetic resonance imaging (MRI) is essential for visualizing the healthy and diseased brain, yet the cellular basis of MRI signal and how it changes over time remain poorly understood. Here, we present BRIDGE (Brain Radiological Imaging with Deep-learning based Ground-Truth Exploration), a platform integrating in vivo MRI with in vivo two-photon (2P) and ex vivo super-resolution microscopy using a multi-step, iterative co-registration pipeline. It enables in vivo, longitudinal, and voxel-precise mapping of MRI signals to their cellular origins for the first time. The registered overlay reveals the cellular and anatomical origins of MRI signals and enables training of convolutional neural networks to enhance the effective resolution of MRI. Using BRIDGE, we identified a microenvironmental vessel biomarker for early metastatic colonization in patient-derived xenograft models of brain metastasis. In particular we found that distinct T2*-weighted hypointense lesions correspond to reduced blood flow and erythrostasis in perimetastatic capillaries. In glioma, longitudinal intravital studies further demonstrated direct correlations between non-vasogenic T2-weighted signal changes and patient-dependent tumor growth dynamics. Taken together, BRIDGE advances radiological interpretation by establishing a microscopic ground truth for MRI signatures over time, enabling deep learning-based predictive histology, and providing cellular-level insights into tumor microenvironment features with direct clinical imaging implications. Graphical abstractBRIDGE enables longitudinal voxel-to-voxel correlation and ground truth based automatic segmentation of MR images O_FIG O_LINKSMALLFIG WIDTH=177 HEIGHT=200 SRC="FIGDIR/small/680637v1_ufig1.gif" ALT="Figure 1"> View larger version (73K): org.highwire.dtl.DTLVardef@f1f64eorg.highwire.dtl.DTLVardef@1619e3eorg.highwire.dtl.DTLVardef@1dc2e7forg.highwire.dtl.DTLVardef@7097e1_HPS_FORMAT_FIGEXP M_FIG C_FIG

The lifetime of fluorescent molecules provides an orthogonal readout to fluorescence intensity, opening experimental possibilities of measuring changes in local molecular environments, mechanical tension, and metabolism, among other factors. These changes are best studied live and in vivo; however, limitations of slow imaging speeds, high phototoxicity, and increased data size and complexity have significantly impeded progress on this front. Here, we present a complete and transferable pipeline consisting of a light sheet FLIM microscope and an accompanying machine learning model for data processing that renders long-term and/or high-speed volumetric FLIM (vFLIM) tractable in living systems. We benchmark this pipeline across several biological use cases, model systems, lifetime ranges, and spatiotemporal scales, showcasing a suite of possibilities that our workflow enables. This comprehensive pipeline from imaging to analysis is a crucial step forward towards disseminating the power of live vFLIM to the broader bioimaging community.

Deep learning provides an opportunity to automatically segment and extract cellular features from high-throughput microscopy images. Many labeling strategies have been developed for this purpose, ranging from the use of fluorescent markers to label-free approaches. However, differences in the channels available to each respective training dataset make it difficult to directly compare the effectiveness of these strategies across studies. Here we explore training models using subimage stacks composed of channels sampled from larger, hyper-labeled, image stacks. This allows us to directly compare a variety of labeling strategies and training approaches on identical cells. This approach revealed that fluorescence-based strategies generally provide higher segmentation accuracies but were less accurate than label-free models when labeling was inconsistent. The relative strengths of label and label-free techniques could be combined through the use of merging fluorescence channels and out-of-focus brightfield images. Beyond comparing labeling strategies, using subimage stacks for training was also found to provide a method of simulating a wide range of labeling conditions, increasing the ability of the final model to accommodate a greater range of experimental setups.

Cryo-electron tomography (CET) combined with sub-volume averaging (SVA), is currently the only imaging technique capable of determining the structure of proteins imaged inside cells at molecular resolution. To obtain high-resolution reconstructions, sub-volumes containing randomly distributed copies of the protein of interest need be identified, extracted and subjected to SVA, making accurate particle detection a critical step in the CET processing pipeline. Classical template-based methods have high false-positive rates due to the very low signal-to-noise ratios (SNR) typical of CET volumes, while more recent neural-network based detection algorithms require extensive labeling, are very slow to train and can take days to run. To address these issues, we propose a novel particle detection framework that uses positive-unlabeled learning and exploits the unique properties of 3D tomograms to improve detection performance. Our end-to-end framework is able to identify particles within minutes when trained using a single partially labeled tomogram. We conducted extensive validation experiments on two challenging CET datasets representing different experimental conditions, and observed more than 10% improvement in mAP and F1 scores compared to existing particle picking methods used in CET. Ultimately, the proposed framework will facilitate the structural analysis of challenging biomedical targets imaged within the native environment of cells.

In standard histopathology, hematoxylin and eosin (H&E) staining stands as a pivotal tool for cancer tissue analysis. However, this method is limited to two-dimensional (2D) analysis or requires labor-intensive preparation for three-dimensional (3D) inspection of cancer tissues. In this study, we present a method for 3D virtual H&E staining of label-free cancer tissues, employing holotomography and deep learning. Holotomography is used to measure the 3D refractive index (RI) distribution of the label-free cancer slides. A deep learning-based image-to-image translation framework is integrated into the resulting 3D RI distribution, enabling virtual H&E staining in 3D. Our method has been applied to colon cancer tissue slides with thicknesses up to 20 m, with conventional chemical H&E staining providing a direct validation for the method. This framework not only bypasses the conventional staining process but also provides 3D structures of glands, lumens, and individual nuclei. The results demonstrate enhancement in histopathological efficiency and the extension of the standard histopathology into the 3D realm. To validate the repeatability and scalability of the approach, we applied the framework to the gastric cancer slides obtained from different institute and imaging devices.

Virtual staining provides a non-invasive alternative to fluorescence microscopy, yet existing deep learning approaches fundamentally rely on pixel-aligned, multiplexed fluorescence targets for supervision. This dependence on rigidly paired data limits scalability, constrains flexibility in generating diverse subcellular structures, and becomes impractical in data-scarce biological settings. In this work, we introduce a semantic supervision paradigm for virtual staining, demonstrating that domain-knowledge prompts can effectively replace conventional pixel-level supervision. Unlike existing methods constrained by rigidly paired multiplex targets, our framework leverages biological prompts to decouple structural guidance from image translation. This decoupling enables high-fidelity, independent synthesis of multiple subcellular structures using only single-channel data. To ensure high-fidelity generation under weak supervision, we integrate self-supervised representation learning to mitigate data scarcity and incorporate direct preference optimization to suppress structural artifacts. Evaluations on the JUMP benchmark demonstrate that our approach effectively balances flexibility and fidelity, outperforming supervised baselines with a 43.3 % reduction in Average FID and an Average PCC of 0.912, while exhibiting high robustness in channel-deficient scenarios. Furthermore, the model generalizes across four in-house datasets to successfully multiplex six subcellular structures, overcoming the physical constraints of conventional fluorescent staining.

Fluorescence lifetime imaging microscopy (FLIM) provides molecular contrast that is largely independent of fluorophore concentration, yet it remains constrained by a persistent trade-off among acquisition speed, photon dose, and detector complexity. To address this challenge, we developed image-projection fluorescence lifetime imaging microscopy (IP-FLIM), an integrated optical and computational platform that enables high-resolution, component-resolved lifetime imaging using only a linear single-photon avalanche diode array. We validate IP-FLIM using fluorescent microbeads and bovine pulmonary artery endothelial cells, demonstrating up to 22.3x improvement in contrast-to-noise ratio and 72.3% reduction in background noise over conventional filtered back-projection reconstruction. By combining wide-field projection acquisition with computational k-space reconstruction, IP-FLIM provides a scalable route to fast, high-resolution multiplex lifetime imaging.

Cell morphology reflects cell health and can distinguish cell-cycle stage, growth arrest, and distinct pathways of cell death. Live, label-free quantitative phase imaging (QPI) captures these features non-invasively and with high temporal resolution, yet many image-based classifiers rely on single frames and cannot separate states whose differences emerge only over time. How much temporal information is needed, and which architecture best exploits it, remain open questions. We assembled 1,874 QPI timelapse sequences spanning six cell states (interphase, mitosis, cell cycle arrest, apoptosis, ferroptosis, and necroptosis) and compared two-dimensional convolutional neural networks (CNNs) with a three-dimensional (3D) spatiotemporal CNN across increasing frame counts. Accuracy improved as frames were added, with the largest gain between one and three frames. The 2D models saturated beyond three frames, whereas the 3D architecture kept improving, reaching 96.5% accuracy and a 3.5% error rate at eleven frames. The temporal information needed tracked the timescale of each process: mitosis was resolved from a single frame, while ferroptosis benefited most from extended sequences. Overall, these results show that dynamic information, rather than static morphology alone, drives accurate cell-state classification, and that 3D architectures are needed to fully exploit it for label-free dynamic phenotyping.

Advancements in transmission electron microscopy (TEM) have enabled in-depth studies of biological specimens, offering new avenues to large-scale imaging experiments with subcellular resolution. Mitochondrial structure is of growing interest in cancer biology due to its crucial role in regulating the multi-faceted functions of mitochondria. We and others have established the crucial role of mitochondria in triple-negative breast cancer (TNBC), an aggressive subtype of breast cancer with limited therapeutic options. Building upon our previous work demonstrating the regulatory role of mitochondrial structure dynamics in the metabolic adaptations and survival of chemotherapy-refractory TNBC cells, we sought to extend those findings to a large-scale analysis of transmission electron micrographs. Here we present a U-Net artificial intelligence (AI) model for automatic annotation and assessment of mitochondrial morphology and feature quantification. Our model is trained on 11,039 manually annotated mitochondria across 125 micrographs derived from a variety of orthotopic patient-derived xenograft (PDX) mouse model tumors and adherent cell cultures. The model achieves an F1 score of 0.85 on test micrographs at the pixel level. To validate the ability of our model to detect expected mitochondrial structural changes, we utilized micrographs from mouse primary skeletal muscle cells genetically modified to lack Dynamin-related protein 1 (Drp1). The algorithm successfully detected a significant increase in mitochondrial elongation, in alignment with the well-established role of Drp1 as a driver of mitochondrial fission. Further, we subjected in vitro and in vivo TNBC models to conventional chemotherapy treatments commonly used for clinical management of TNBC, including doxorubicin, carboplatin, paclitaxel, and docetaxel (DTX). We found substantial within-sample heterogeneity of mitochondrial structure in both in vitro and in vivo TNBC models and observed a consistent reduction in mitochondrial elongation in DTX-treated specimens. We went on to compare mammary tumors and matched lung metastases in a highly metastatic PDX model of TNBC, uncovering significant increase in mitochondrial length in metastatic lesions compared to their cognate mammary tumor. This dataset provides high statistical power to detect frequent chemotherapy-induced shifts in mitochondrial shapes and sizes in residual cells left behind after treatment. The successful application of our AI model to capture mitochondrial structure marks a step forward in high-throughput analysis of mitochondrial structures, enhancing our understanding of how morphological changes may relate to chemotherapy efficacy and mechanism of action. Our large, manually curated electron micrograph dataset - now publicly available - serves as a unique gold-standard resource for developing, benchmarking, and applying computational models, while further advancing investigations into mitochondrial morphology and its impact on breast cancer biology.

Single-Molecule Localization Microscopy remains limited in its ability for robust and simple multi-color imaging. Whereas the fluorophore Alexa647 is widely used due to its brightness and excellent blinking dynamics, other excellent blinking fluorophores, such as CF660 and CF680, spectrally overlap. Here we present Probability-based Fluorophore Classification, a method to perform multi-color SMLM with Alexa647, CF660 and CF680 that uses statistical decision theory for optimal classification. The emission is split in a short and long wavelength channel to enable classification and localization without a major loss in localization precision. Each emitter is classified using a Generalized Maximum Likelihood Ratio Test using the photon statistics of both channels. This easy-to-adopt approach does not require nanometer channel registration, is able to classify fluorophores with tunable low false positive rates (<0.5%) and optimal true positive rates and outperforms traditional ratiometric spectral de-mixing and Salvaged Fluorescence. We demonstrate its applicability on a variety of samples and targets.

Localization microscopy techniques have overcome the diffraction limit, enabling nanoscale biological imaging by precisely determining the positions of individual emitters. However, the performance of deep learning methods commonly applied to these tasks often depends significantly on the quality of training data, typically generated through simulation. Creating simulations that perfectly replicate experimental conditions remains challenging, resulting in a persistent simulation-to-experiment (sim2exp) gap. To bridge this gap, we propose a physics-informed generative model leveraging self-supervised learning directly on experimental data. Our model extends the Deep Latent Particles (DLP) framework by incorporating a physical Point Spread Function (PSF) model into the decoder, enabling it to disentangle learned realistic environments from precise emitter properties. Trained directly on unlabeled experimental images, our model intrinsically captures realistic background, noise patterns, and emitter characteristics. The decoder thus acts as a high-fidelity generator, producing fully labeled, realistic training images with known emitter locations. Using these generated datasets significantly improves the performance of supervised localization algorithms, particularly in challenging scenarios such as complex backgrounds and low signal-to-noise ratios. Our results demonstrate substantial improvements in localization accuracy and emitter detection, underscoring the practical benefit of our approach for real-world microscopy applications. We will make our code publicly available.

Fluorescence Lifetime Imaging Microscopy (FLIM) quantifies the autofluorescence lifetime to measure cellular metabolism, therapeutic efficacy, and disease progression. These dynamic processes are intrinsically heterogeneous, increasing the complexity of the signal analysis. Often noise reduction strategies that combine thresholding and non-selective data smoothing filters are applied. These can result in error introduction and data loss. To mitigate these issues, we develop noise-corrected principal component analysis (NC-PCA). This approach isolates the signal of interest by selectively identifying and removing the noise. To validate NC-PCA, a secondary analysis of FLIM images of patient-derived colorectal cancer organoids exposed to a range of therapeutics was performed. First, we demonstrate that NC-PCA decreases the uncertainty up to 4-fold in comparison to conventional analysis with no data loss. Then, using a merged data set, we show that NC-PCA, unlike conventional methods, identifies multiple metabolic states. Thus, NC-PCA provides an enabling tool to advance FLIM analysis across fields.

Mapping peritumoral collagen fiber directionality in solid tumors may assist in determining cancer progression and support more personalized prognoses. However, existing microscopy techniques are often limited by a restricted field of view, high cost, or incompatibility with paraffin-treated tissues. Computational scattered light imaging (ComSLI) is a cost-effective whole-slide microscopy technique that reveals fiber orientations independent of sample preparation. Using glioma, colorectal, and head and neck cancer samples, we show for the first time that ComSLI maps fiber orientations in paraffin-treated tumor tissues, visualizes tumor growth pathways and desmoplastic reactions, and allows the study of collagen orientations relative to tumor boundaries.

Light-sheet fluorescence microscopy (LSFM) has revolutionized biological imaging by enabling high spatial and temporal resolution with minimal photodamage. However, conventional LSFM techniques often suffer from striping artifacts in the resulting images due to light scattering and absorption within samples, leading to uneven illumination that negatively impacts the accuracy of subsequent image analyses. To address this limitation, we introduce dodecagon light-sheet fluorescence microscopy (dodecaLSFM), a novel approach that maximizes angular diversity to achieve homogeneous illumination and suppress striping artifacts. dodecaLSFM employs diffraction optics and cylindrical lenses to generate twelve light sheets, providing 360 degree omnidirectional illumination that significantly enhances illumination uniformity compared to traditional mSPIM, mDSLM, and ultramicroscopy systems, which use only one or two illumination planes. We demonstrate the effectiveness of dodecaLSFM by achieving high-resolution imaging of whole mouse brain vasculature following tissue clearing, allowing precise morphometric analysis of vascular networks without striping artifacts. Furthermore, we show that combining dodecaLSFM with expansion microscopy (ExM) enables whole-organ 3D imaging at cellular resolution. This novel approach provides an advanced, scalable solution for large-volume imaging, facilitating detailed structural and functional studies across diverse biological applications.

Tissue clearing and Light sheet fluorescence microscopy (LSFM) provide spatial information at a subcellular resolution in intact organs and tumors which is a significant advance over tools that limit imaging to a few representative tissue sections. The spatial distribution of drugs, targets, and biomarkers can help inform relationships between exposure at the site of action, efficacy, and safety during drug discovery. We demonstrate the use of LSFM to investigate distribution of an oncolytic virus (OV) and vasculature in xenograft tumors, as well as brain A{beta} pathology in an Alzheimers disease (AD) mouse model. Machine learning-based image analysis tools developed to segment vasculature in tumors showed that random forest and deep learning methods provided superior segmentation accuracy vs intensity-based thresholding. Sub-cellular resolution enabled detection of punctate and diffuse intracellular OV distribution profiles. LSFM investigation in the brain in a TgCRND8 AD mouse model at 6.5 months of age enabled evaluation of A{beta} plaque density in different brain regions. The utility of LSFM data to support quantitative systems pharmacology (QSP) and physiology-based pharmacokinetics (PBPK) modeling to inform drug development are also discussed. In summary, we showcase how LSFM can expand our understanding of macromolecular drug and biomarker distribution to advance drug discovery and development.

Cryo-volume electron microscopy (cryo-vEM) enables near-native visualization of cellular ultrastructure, but its broad use is limited by low image contrast and the high cost of dense voxel-level annotation. Existing automated segmentation methods often generalize poorly across cell types, organelles, and imaging conditions. Here, we introduce SparseSeg, a target-conditioned, sparsity-driven segmentation framework that treats organelle segmentation as a discovery process rather than a closed-set classification task. SparseSeg uses a small number of context-specific exemplars to iteratively propagate reliable supervision through the volume. It combines sparse patch-based sampling, a multi-kernel U-Net, and geometry-consistent refinement to expand accurate segmentation while suppressing context-dependent false positives. Across serial cryo-FIB-SEM and conventional vEM datasets, SparseSeg achieves robust segmentation under extreme sparse annotation, including settings with less than 1% labeled slices. This framework reduces annotation burden while preserving morphological fidelity for quantitative cryo-vEM analysis.

High-speed three-dimensional imaging in scattering tissues remains challenging because volumetric microscopy generally requires scanning, whereas snapshot light-field approaches divide limited detector pixels among multiple views. This constraint is particularly severe in the second near-infrared window (NIR-II), where commonly used InGaAs cameras typically have relatively small sensor formats and high detector noise. Here we introduce NIR-II squeezed light-field microscopy (NIR-II SLIM), which optically rotates and compresses multiple perspective views before detection, allowing efficient use of camera pixels while retaining complementary spatial information for three-dimensional reconstruction. NIR-II SLIM acquires volumes at up to 600 volumes s-1 with a reconstructed lateral sampling grid of 512 x 512 pixels. We use the method for label-free four-dimensional imaging of cardiac dynamics in pigmented late-larval zebrafish, resolving chamber deformation and millisecond-scale atrioventricular-valve motion, and for NIR-II fluorescence imaging of vascular and lymphatic transport in mice. NIR-II SLIM provides a detector-efficient approach for high-speed volumetric imaging of rapid biological dynamics in scattering tissues.

The specimen-induced aberration has been a major factor limiting the imaging depth of single-molecule localization microscopy (SMLM). Here, we report the application of label-free wavefront sensing adaptive optics to SMLM for deep-tissue super-resolution imaging. The proposed system measures complex tissue aberrations from intrinsic reflectance rather than fluorescence emission and physically corrects the wavefront distortion more than three-fold stronger than the previous limit. This enables us to resolve sub-diffraction morphologies of cilia and oligodendrocytes in whole intact zebrafish as well as dendritic spines in thick mouse brain tissues at the depth of up to 102 m with localization number enhancement by up to 37 times and localization precision comparable to aberration-free samples. The proposed approach can expand the application range of SMLM to intact animals that cause the loss of localization points owing to severe tissue aberrations.

Soft X-ray tomography (SXT) is a powerful, non-invasive bio-imaging technique that enables visualization of cellular structures in near-native states. Despite its potential, the development of dedicated image analysis tools -- particularly deep-learning-based models -- has been limited, largely due to the limited accessibility of soft X-ray microscopes and the scarcity of labeled SXT data. To address this deficit, in this work, we present SXTractor, a self-supervised SXT feature extractor based on the DINO framework. SXTractor can be fine-tuned with minimal labeled data and effectively adapted to various downstream tasks. We demonstrate its utility on few-shot tomogram segmentation, where it significantly outperforms the model when trained from scratch. Furthermore, it achieves few-shot segmentation performance comparable to that of the Segment Anything Model (SAM), despite SAM being a segmentation-specific model pretrained on millions of labeled images with a significantly larger model size. Most importantly, SXTractor enables a diverse range of downstream applications of deep learning to SXT, thus offering a practical and scalable solution for SXT image analysis in data-constrained settings.

IntroductionGlioblastoma is characterized by heterogeneous tumor characteristics and infiltrative tumor boundaries, making accurate delineation difficult with extensive manual annotations. Chemical exchange saturation transfer (CEST) is a non-invasive MRI technique used for in vivo assessment of metabolic and macromolecular information through a Z-spectrum. CEST may provide insight into metabolic changes present in early-stage disease that are not visible in routine clinical imaging, thereby improving tumor delineation. In this work, we use an unsupervised anomaly detection (UAD) strategy to learn the distribution of features present in Z-spectra of healthy tissue and capture their deviations in pathology, foregoing the need for extensive labels. The approach leverages the metabolic information provided by CEST to improve the detection and delineation of glioblastoma and inform further treatment planning. MethodsA 1D convolutional autoencoder (CAE) was implemented to reconstruct Z-spectra from individual tissue voxels. The network was trained on Z-spectra acquired at 9.4T from healthy Sprague-Dawley rats and tested on data acquired from F98 glioma-bearing rats post Gd-administration. For baseline comparisons, Isolation Forest and Local Outlier Factor, which have shown success in anomaly detection, were implemented. For the CAE, our anomaly score was determined to be the mean squared reconstruction error. To facilitate clinical translation and evaluate the robustness of our model for under sampled Z-spectra, acceleration factors of 2x and 7x were performed with two sampling schemes: uniformly skipping frequency offsets and selecting offsets based on feature importance identified by Shapley value analysis and Integrated Gradients (IG). Binarization was performed by determining an optimal anomaly threshold, followed by comparison to ground truth tumor masks. Metrics related to model performance were assessed for baseline anomaly detectors on the fully sampled dataset and for the CAE on fully and under sampled datasets. ResultsThe best baseline anomaly detector was Isolation Forest, with an ROC-AUC of 0.967 and an F1-score of 0.584. Our method, the CAE, accurately reconstructed Z-spectral features, achieving Dice scores of up to 0.72 and outperforming the baseline model with an ROC-AUC of 0.968 and F1-score of 0.642. This model performance remained robust across sampling schemes and acceleration factors, with ROC-AUCs of [~]0.96 and similar Dice scores (up to 0.7). Feature importance analysis indicated that offsets in the range of {+/-}3.0 to 5.0ppm contributed most to the anomaly score. DiscussionThis study successfully demonstrated a UAD pipeline utilizing the Z-spectrum from CEST MRI for metabolically informed tumor delineation. The framework captures biochemical deviations that may precede or extend beyond morphologic abnormalities, enabling sensitive detection of tumor regions and intra-tumoral heterogeneity that previous methods may fail to capture. The offsets from the feature analysis indicated a strong contribution from the magnetization transfer (MT) pool to the spectral deviations captured by the model, with additional contributions from relayed nuclear Overhauser effect (rNOE) and amide proton transfer (APT). Model robustness with under sampling further highlights the pipelines potential in accelerated acquisitions, thus improving clinical practicality. While there is a need for validation on larger cohorts and clinical datasets, the current results demonstrate that this label-free, Z-spectral anomaly mapping can serve as an interpretable and scalable tool for monitoring tumor heterogeneity and progression, with potential applicability to other diffuse or metabolically subtle pathologies.